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Abstract 

Information  functions  are  used  to  find  the  optimum  ability 
levels  and  maximum  contributions  to  information  for  estimating  item 
parameters  in  three  commonly  used  logistic  item  response  models. 

For  the  three  and  two  parameter  logistic  models,  examinees  who 
contribute  maximally  to  the  estimation  of  item  difficulty  contribute 
little  to  the  estimation  of  item  discrimination.  This  suggests  that 
in  applications  that  depend  heavily  upon  the  veracity  of  individual 
item  parameter  estimates  (e.g.  adaptive  testing  or  test 
construction),  better  item  calibration  results  may  be  obtained  (for 
fixed  sample  sizes)  from  examinee  calibration  samples  in  which 
ability  is  widely  dispersed. 
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Introduction 

The  success  of  applications  in  Item  Response  Theory  (IRT) 
depends  upon  the  accuracy  with  which  individual  item  parameters  can 
be  estimated.  This  dependence  is  especially  important  for  those 
applications  that  depend  heavily  upon  the  veracity  of  individual 
item  parameter  estimates  in  contexts  perhaps  unrelated  to  the 
calibration  setting  in  which  estimates  were  obtained.  Two  recent 
examples  of  such  applications  are  adaptive  testing  (see,  for 
example,  Lord  (1980,  chap.  10);  Stocking  (1988))  and  IRT-based  test 
development  (see,  for  example,  van  der  Linden  &  Boekkooi -Timminga 
(1987)). 

Typically,  calibration  samples  are  selected  with  an  eye  to 
convenience  as  random  subsaraples  from  a  larger  set  of  data  (see,  for 
example.  Cook,  Petersen,  &  Stocking  (1983)).  It  may  be  possible  to 
obtain  better,  i.e.,  more  accurate,  item  parameter  estimates  if  the 
planning  for  calibration  sample  selection  explicitly  includes 
considerations  of  the  accuracy  of  the  resulting  item  parameter 
estimates.  This  idea  was  suggested,  based  on  the  analogy  between 
item  parameter  estimation  and  estimation  problems  in  regression,  20 
years  ago  by  Lord  (1968).  More  recently,  Wingersky  and  Lord  (1984) 
provide  theoretical  support. 
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One  approach  to  issues  of  the  accuracy  of  item  parameter 
estimation  considers  the  asymptotic  expected  covariance  matrix. 

Lord  (1980,  p.  191)  presents  formulas  from  which  this  matrix  can  be 
derived  for  the  three-parameter  logistic  (3PL)  model  in  the  finite 
sample  case  with  known  abilities.  Thissen  and  Wainer  (1982) 
consider  the  properties  of  the  asymptotic  covariance  matrix  for  item 
parameters  when  a  particular  distribution  of  ability  is  assumed  in 
the  calibration  example.  Lord  and  Wingersky  (1985)  consider  the 
asjrmptotic  properties  of  the  covariance  matrix  for  the  3PL  model 
when  both  item  parameters  and  abilities  are  estimated  jointly  by 
maximum  likelihood. 

In  this  paper  we  explore  the  relationship  between  examinee 
ability  and  the  accuracy  of  maximum  likelihood  item  parameter 
estimation  in  terms  of  the  expected  (Fisher)  information  (Kendall  & 
Stuart,  1979,  p.  10).  The  advantage  to  this  approach  is  that  it  is 
possible  to  study  directly  the  relationship  between  examinee  ability 
and  contribution  to  information  in  some  detail  and  with  some 
surprising  results.  This  theoretical  examination  depends  upon  the 
assumption  that  examinee  abilities,  as  well  as  some  item  parameters, 
are  known.  However,  the  results  can  be  translated  into  general 
guidelines  for  situations  in  which  samples  can  be  selected  on  the 
basis  of  some  observed  score  for  the  purpose  of  obtaining  accurate 
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parameter  estimates  for  a  collection  of  items  whose  properties  are 
imperfectly  known. 

The  Theory 

Suppose  the  probablility  of  a  correct  response  to  an  item  is 
specified  by  a  logistic  function,  either  one  parameter  logistic 
(IPL),  two  parameter  logistic  (2PL)  or  three  parameter  logistic 
(3PL).  Suppose  further  that  estimates  of  the  item  parameters  are 
obtained  by  maximum  likelihood.  If  the  model  fits  the  data,  and 
true  abilities  are  known,  then  formulas  exist  for  the  amount  of 
information  in  the  sample  for  estimating  item  parameters  (Lord, 

1980,  eqs .  12-8  through  12-13)). 

Of  course,  the  model  never  fits  any  set  of  real  data,  and  true 
abilities  are  never  known.  Therefore,  the  information  computed  from 
the  Lord  formulas  is  an  estimate  of  the  maximum  possible 
information,  and  will  be  larger  than  what  can  be  realistically 
obtained  (Mislevy  &  Sheehan,  1987).  Nevertheless,  an  examination  of 
what  the  theory  predicts  may  be  useful  in  planning  a  calibration 
sample  to  estimate  item  parameters  accurately. 

If  the  probability  of  a  correct  response  by  examinee  a  to  an 
item  is  P  ,  the  log  of  the  likelihood  of  observed  responses  to  the 

cL 

item  for  N  examinees  is 
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N 

i  -  2  [u  inP  +  (1  -  u  )inQ  ] 

^  Si  Si  Si  Q. 

a-1 

where  u  =  0  if  the  response  in  incorrect,  u  -  1  if  the  response 
Si  a 

is  correct,  and  Q  -  1  -  P  .  The  maximum  likelihood  estimates  of 
a  a 

each  item  parameter  x  located  at  a  point  where  the  partial 
derivatives  of  the  log  likelihood  are  zero.  The  expected  second 
partial  derivatives  of  the  log  likelihood  can  be  computed  at  this 
point.  (The  inverse  of  the  negative  of  this  matrix  is  the 
asymptotic  variance/covariance  matrix  used  by  Thissen  and  Wainer 
(1982).)  The  negative  of  a  diagonal  element  of  the  matrix  of 
expected  second  partial  derivatives  is  referred  to  as  the 
information  in  the  sample  for  estimating  an  item  parameter.  For  any 
item  parameter  Xt  this  information  function  has  the  form  (Lord, 

1980,  equations  12-8  to  12-13) 


I 

XX 


N 

2 

a~l 


_i —  ^  — a  ^2 

a  a 


N 

2 

a-1 


The  information  is  composed  of  individual  additive  contributions 


from  each  examinee. 
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i 

XX 


_1 

PQ 


^  .2 
^X  ^ 


(1) 


where  the  subscript  a  has  been  dropped  for  convenience .  By 
examining  the  properties  of  an  individual  contribution  as  a  function 
of  ability,  we  can  determine  what  values  of  ability  provide  the  most 
(and  least)  information  for  estimating  an  item  parameter,  assuming 
other  item  parameters  are  fixed. 

Results 


The  3 PL  Model 


The  3PL  item  response  function  is 


P 


c  + 


_ 1  -  c _ 

1  ^  ,-Da((^-b) 


where 

a  is  a  function  of  the  slope  of  the  item  response  function 
in  the  neighborhood  of  the  item  difficulty; 

b  characterizes  the  difficulty  or  location  of  the  item  on 
the  ability  continuum; 

c  is  the  lower  bound  of  probabilities  of  correct  response, 
even  from  low  ability  examinees; 

6  is  examinee  ability; 
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D  is  a  scaling  constant  commonly  used  with  the  value  of  1.7. 

The  partial  derivatives  of  P  with  respect  to  the  item 
parameters  are  given  by  Lord  (1980,  Equation  12-2).  Substituting 
the  derivative  with  respect  to  c  into  Equation  1  gives  an  examinee's 
contribution  to  the  information  for  estimating  c  as 


i 

cc 


Q 

p 


(2) 


Values  of  ability  that  give  local  minima  and  maxima  of  Equation  2 
are  found  where 


di 

cc 

de 


-Da 


Q(P  -  c)  =  0 
P 


This  derivative  is  zero  when  Q  -  0,  in  which  case  ^  =  -Ho  and  i 

cc 

is  zero.  It  is  also  zero  when  P  =  c,  in  which  case  =  -»  and 

i  -  - - -  ~  .  This  latter  value  forms  an  upper  asymptote  to 

the  amount  of  information  from  all  ability  levels.  The  higher  the 
value  of  c,  the  more  examinees,  even  optimal  examinees,  are  required 


for  estimation  of  c. 
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The  information  for  estimating  b  is  obtained  by  substituting 
the  derivative  of  P  with  respect  to  b  into  Equation  1,  giving 


D^a^  .20 

^bb  -  - - P 

(1  -  c) 


(3) 


with  partial  derivative 


^  -  aiP-:-^  [.2p2  +  P  +  c] 


(4) 


A  root  of  this  derivative  is  found  where  Q  -  0,  in  which  case 
^  and  i,,  -  0.  A  second  root  is  found  when  P  -  c,  where 

DO 

$  -  -at  and  i^j^  -  0.  Examinees  with  abilities  far  away  from  the 

item's  location  are  useless  for  estimating  that  location.  A  final 

2 

root  of  this  derivative  is  found  where  -2P  +P+c=0.  The  root 

of  this  latter  quantity,  P*,  where  c  <  P*  <  1 ,  is 

P*  —  1^jLj/1_jL_8c  optimal  ability  9*  is  then 


9* 


in( 


P* 

1  - 


c 

P* 


) 


(5) 
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and 


n2  2 

D  a 


bb 


(1  -  c)' 


(P*  -  c)' 


Q* 

p* 


(6) 


where  Q*  =  1  -  P*. 

The  optimal  ability  level  is  at  =  b  when  c  -  0.  When 
0  <  c  <  1,  the  optimal  ability  level  is  greater  than  b  by  an  amount 
that  depends  upon  both  a  and  c.  For  a  fixed  value  of  c,  higher 
values  of  a  mean  that  the  optimum  location  is  closer  to  but  still 
greater  than  b.  Since  P*  is  a  function  of  c  alone,  the  maximum 
amount  of  information  from  an  examinee  for  estimating  b,  Equation  6, 
depends  only  on  a  and  c,  and  for  fixed  c,  is  proportional  to  the 
square  of  a. 

By  analagous  computations,  the  information  from  each  examinee 
(Equation  1)  for  estimating  a  for  the  3PL  is 


i 

aa 


(1  -  c) 


2 


(P  -  c) 


2  2 
P 


(7) 


with  partial  derivative 
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3i 


aa 


de 


(1  -  c)‘ 


b)(p  -  cro 


[(e  -  b) 


Da 


(1  -  c) 


(-2P 


+  P  +  c) 

(8) 


+  2P] 


Three  roots  of  Equation  8  are  at  5  =  b,  6  -  -«>,  and  8  =  +«>.  For 
all  three  of  these  roots,  i  “0.  Low  and  high  ability 

3i8i 

examinees,  as  well  as  examinees  whose  abilities  may  be  close  to 
optimum  for  estimating  b  are  of  no  use  in  the  estimation  of  a.  The 
location  of  optimal  abilities  is  found  from  the  last  factor  in 
Equation  8.  This  expression  is  zero  when  the  optimal  ability 


S** 


b  + 


2(1  -  c^P** 


Da(2P**  -  P**  -  c) 


(9) 


The  optimal  ability  is  not  given  explicitly  by  this  expression,  but 

values  of  8**  can  be  found  by  numerical  methods.  The  top  panel  of 

Figure  1  shows  P  plotted  as  a  function  of  ^  -  b,  from  Equation  9. 

1  2 

When  P=c,  8  -b=-r~,  and  when  P=l,  8  -b-~.  The  plot 

Da  Da 

suggests  that  these  values  form  upper  and  lower  bounds  respectively 
for  two  reprons  in  which  the  optimal  ability  might  be  found: 

$  <  b  -  and  8  >  h  +  . 
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Insert  Figure  1  about  here 


Numerical  Examples 

Equations  2,  3,  and  7  are  plotted  as  functions  of  ability  in 
the  top  panel  of  Figure  2  for  an  item  with  a  -  1,  b  -  0,  and  c  -  .2. 
Additive  contributions  to  information  for  estimating  a  or  b  from 
examinees  at  various  ability  levels  must  be  read  from  the  right-hand 
scale;  those  for  estimating  c  must  be  read  from  the  left-hand  scale. 
Figure  2  demonstrates  graphically  what  we  have  learned  analytically; 
1)  the  contribution  of  examinees  to  the  information  available  for 
estimating  item  difficulty  is  asymetric  around  b,  with  higher 
ability  examinees  contributing  more  information;  2)  examinees  who 
are  most  informative  in  the  estimation  of  item  difficulty  are  of 
little  use  in  estimating  item  discrimination;  3)  examinees  who  do 
contribute  to  the  estimation  of  discrimination  are  asymetrically 
distributed  around  the  item  difficulty;  and  U)  only  low  ability 
examinees  contribute  much  information  for  the  estimation  of  c,  and 
there  is  a  limit  to  this  contribution. 


Insert  Figure  2  about  here 
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The  effects  of  different  values  of  a,  for  fixed  b  and  c,  on  the 
additive  contribution  of  examinees  to  information  for  estimating  a 
are  shown  in  the  top  left  panel  of  Figure  3.  Each  examinee 
contributes  substantially  more  information  to  the  estimation  of  a  if 
the  value  of  a  is  low  rather  than  high.  More  lower  ability 
examinees  than  higher  ability  examinees  are  required  to  obtain  a 
given  amount  of  information  for  the  estimation  of  a,  regardless  of 
the  value  of  a,  since  lower  ability  examinees  contribute  less 
information  than  higher  ability  examinees. 


Insert  Figure  3  about  here 


The  middle  left  panel  of  Figure  3  shows  the  effects  of  different 
values  of  b,  for  fixed  a  and  c,  on  the  additive  contribution  of 
examinees  to  information  for  estimating  b.  Changing  b  shifts  the 
location  of  optimal  examinees,  but  not  the  maximum  contribution. 

The  effects  of  different  c's  on  the  additive  contribution  for 
estimating  c,  for  fixed  a  and  b,  are  shown  in  the  bottom  left  panel 
of  Figure  3.  For  lower  ability  levels,  the  higher  the  guessing 
parameter,  the  less  the  additive  contribution  for  each  examinee; 
more  examinees  are  required  to  obtain  a  given  amount  of  information 
about  c.  Regardless  of  the  value  of  c,  higher  ability  examinees  do 


1 
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not  contribute  much.  However,  they  contribute  more  if  c  is  higher 
than  if  c  is  lower. 

Numerical  values  of  optimal  abilities  and  maximal  contributions 
to  information  (in  parentheses)  are  presented  for  typical  values  of 
a  and  b  for  c  -  .2  in  Table  1.  Optimal  abilities  and  maximal 
contributions  for  b  come  directly  from  Equations  5  and  6.  Those  for 
a  come  from  the  application  of  numerical  methods  to  Equations  9  and 
7. 


Insert  Table  1  about  here 


The  2PL  and  IPL  Models 

The  2PL  item  response  function  can  be  considered  as  a  special 
case  of  the  3PL  with  c  -  0.  In  this  case, 

p  _  - - 

1  ^  ,-Da(fi-b)  • 

The  IPL  ^ tern  response  function  is  also  a  special  case  of  the  3PL 
with  c  -  0  and  constant  a.  The  expression  for  P  is  identical  to 
that  for  the  2PL  immediately  above,  but  the  item  discrimination  is 


constant  for  all  items. 
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The  information  for  estimating  b  for  the  2PL  is  algebraically 
the  same  as  for  the  IPL: 


2  2 

ibb  -  D  a  PQ  . 

with  derivative  with  respect  to  ability 

-  D^a^PQd  -  2P) 


(10) 


These  are  obtained  form  Equations  3  and  4  for  the  3PL  by  setting 

c  -=  0.  Examinees  with  abilities  of  +®  contribute  nothing  to  the 

estimation  of  b  in  these  models.  Examinees  with  =  b  (P*  =  .5) 

2  2 

contribute  the  maximum  amount  of  information,  .25D  a  .  The  additive 
contribution  from  each  examinee  depends  upon  the  square  of  the 
discrimination  of  the  item. 

For  the  2PL  model,  the  information  for  estimating  a  is 

i  -  -  b)^PQ  (11) 

aa 


with  derivative 
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ai 

aa 

89 


=  D^(e  -  b)PQ[Da(5  -  b)(l  -  2P)  + 


2] 


again  obtained  from  the  3PL  results  (Equations  7  and  8)  by  setting 
c  —  0.  As  with  the  3PL,  examinees  with  low  and  high  abilities,  as 
well  as  examinees  whose  abilities  are  optimal  for  estimating  b  are 
useless  for  estimating  a.  The  location  of  optimal  abilities  is 
found  by  setting  Ua{6  •  b)(l  -  2P)  +  2  =  0  in  which  case  the  optimal 
ability  6**  is 


9**  =  b 


Da(2P  -  1) 


(12) 


The  bottom  panel  of  Figure  1  shows  P  plotted  as  a  function  of  S  -  b 

2 

for  the  2PL  in  Equation  12.  When  P=0,  S-b--  “  ,  and  when 

Da 

2 

P“l,  S-b=“.  As  with  the  3PL,  there  are  two  regions  in  which 

2  2 

9**  might  be  found:  <>  <  b  -  ~  and  5  >  b  +  ~  .  In  contrast  to 

Da  Da 

the  3PL,  these  regions  are  symmetric  around  9  -  b  -  0. 

The  additive  contribution  to  estimating  a  for  optimal  examinees 
is 


2  2  1 
i  -  .25D  ($**  -  b)  -  . 

aa  2 

a 


(13) 


Specifying  Optimum  Examinees 

17 

Both  terms  in  Equation  13  decrease  for  higher  values  of  a  since  the 
optimal  8  is  closer  to  the  difficulty  for  Increased  a,  given  fixed 
b.  In  contrast  with  the  3PL,  the  contributions  of  examinees  whose 
true  abilities  are  symmetric  around  the  item  difficulty  are 
identical . 

Numerical  Examples 

Equation  10  for  the  2PL  and  IPL  and  Equation  11  for  the  2PL  are 
plotted  as  fuctions  of  ability  in  the  bottom  panel  of  Figure  2  for 
an  item  with  a  -  1  and  b  -  0.  Additive  contributions  to  Information 
for  estimating  a  (for  fixed  b)  or  b  (for  fixed  a)  are  higher  in 
models  that  do  not  contain  a  guessing  parameter.  This  is  seen  by 
comparing  the  two  panels  in  Figure  2. 

The  effects  of  different  a  values  (with  fixed  b)  on  the 
additive  contributions  for  estimating  a  in  the  2PL  (Equation  11)  are 
shown  in  the  top  right  panel  of  Figure  3.  Examinees  symmetrically 
located  around  the  item  difficulty  contribute  equally  to 
information.  As  in  the  3PL  (top  left  panel),  each  examinee 
contributes  more  to  information  for  estimating  a  when  a  is  low.  In 
contrast  to  the  3PL,  examinee's  additive  contributions  are  higher 
when  the  model  does  not  contain  a  guessing  parameter  (compare  the 
two  top  panels  of  Figure  3).  Equation  10  for  the  2PL  with  fixed  a 
or  the  IPL  is  plotted  for  different  values  of  b  in  the  middle  right 
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panel  of  Figure  3.  As  with  the  3PL  (middle  left  panel),  changing 
values  of  b  shifts  the  location  of  optimal  examinees  but  not  the 
maximum  contribution  to  information  for  estimating  item  difficulty. 

Numerical  values  of  optimal  abilities  and  maximum  contributions 
to  information  (in  parentheses)  are  presented  for  typical  values  of 
a  and  b  for  the  2PL  and  IPL  in  Table  2.  Optimal  abilities  and 
maximum  contributions  for  estimating  b  come  directly  from  Equation 
10  and  its  consequences.  Comparable  data  for  information  for 
estimating  a  for  the  2PL  comes  from  the  application  of  numerical 
methods  to  Equation  12  and  Equation  13. 


Insert  Table  2  about  here 


Discussion 

The  focus  of  this  paper  has  been  to  explore  the  contribution  of 
examinees  with  different  true  abilities  to  the  expected  information 
available  for  estimating  item  parameters.  This  is  not  the  same  as 
an  exploration  of  the  accuracy  with  which  item  parameters  may  be 
estimated,  but  has  the  advantage  of  directly  suggesting  strategies 
of  calibration  sample  selection.  In  terms  of  making  statements 
about  the  accuracy  with  which  item  parameters  are  estimated,  the 
analyses  presented  here  are  immediately  relevant  to  the  problem  of 


I 
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estimating  a  single  item  parameter,  conditional  on  other  item 
parameters  being  fixed  at  their  true  values.  However,  statements 
about  the  accuracy  of  item  parameter  estimates  are  usually  made  in  a 
broader  context  in  which  all  three  item  parameters  must  be  estimated 
and  a  particular  distribution  of  true  ability  may  be  assumed.  Such 
considerations  then  take  into  account  the  covariances  among  the  item 
parameters,  and  depend  upon  the  assumed  distribution  of  ability. 

The  results  need  not  hold  for  different  ability  distributions. 

As  an  example,  the  results  presented  here  show  that  in  both  the 
2PL  and  3PL,  optimal  examinees  contribute  more  information  for 
estimating  item  discrimination  when  that  discrimination  is  low. 

This  cannot  be  interpreted  to  mean  that  low  a's  are  necessarily  more 
accurately  estimated  than  high  a's,  for  a  fixed  sample  size  or  a 
fixed  distribution  of  ability.  Thissen  and  Wainer  (1982),  using  the 
expected  asymptotic  variance/covariance  matrix  and  assuming  a  normal 
distribution  of  ability,  show  that,  for  fixed  b,  the  asymptotic 
standard  error  of  a  decreases  with  a  for  the  2PL.  However,  for 
fixed  b  in  the  3PL  model,  the  asymptotic  error  of  a  is  larger  for 
both  small  and  large  values  of  a  than  it  is  for  more  moderate  values 
of  a.  These  results  depend  on  the  assumed  distribution  of  ability 
as  well  as  the  covariances  among  the  item  parameters. 
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As  an  illustration  of  this  phenomenon,  Figure  4  shows  the 
Thissen  and  Wainer  asymptotic  standard  error  of  b  plotted  against  b 
for  the  3PL  (solid  cuirve) .  This  is  a  partial  reproduction  of 
Thissen  and  Wainer 's  Figure  1,  with  a  -  1.5,  c  -  0.,  and  a  standard 
normal  calibration  sample  of  N  -  2500.  The  results  presented  here 
suggest  that  a  calibration  sample  with  a  larger  spread  of  abilities 
will  improve  the  accuracy  of  the  estimation  of  high  and  low 
difficulties.  Figure  4  shows  the  same  information  for  the  same  item 
when  a  calibration  sample  of  the  same  size  is  drawn  from  a  normal 
distribution  of  ability  with  a  variance  of  4  (dashed  line) . 

Although  the  sample  si^e  is  the  same,  the  accuracy  of  estimation  is 
much  improved  for  the  more  extreme  difficulties. 


Insert  Figure  4  about  here 


Given  the  knowledge  of  the  location  of  optimal  examinees  for 
estimating  the  parameters  for  a  single  item  in  three  different  item 
response  models  presented  here,  what  suggestions  can  we  make  for 
calibration  sample  selection  that  would  aid  practitioners  who  must 
jointly  estimate  the  item  parameters  for  many  items  simultaneously? 
If  the  practitioner  chooses  the  IPL  as  the  appropriate  model,  then 
examineees  whose  true  ability  is  equal  to  item  difficulty  are  most 
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informative.  If  a  collection  of  items  to  be  calibrated  is  thought 
to  have  a  broad  spread  of  difficulties,  based  perhaps  on  the 
conventional  proportions  correct,  then  the  distribution  of  true 
abilities  in  the  calibration  sample  should  also  be  broad.  Such  a 
sample  could  possibly  be  selected  based  on  some  available  observed 
auxiliary  information.  If  the  range  of  abilities  is  too  small,  the 
sample  will  provide  information  only  for  middle  difficulty  items; 
little  information  will  be  provided  to  estimate  the  difficulty  for 
easy  and  hard  items. 

If  the  2PL  is  the  appropriate  model,  estimation  of  both  a  and  b 
requires  a  wider  range  of  true  abilities  than  for  the  IPL.  This  is 
so  because  only  examinees  with  ability  not  equal  to  b  are 
informative  in  the  estimation  of  a.  If  the  range  of  abilities  is 
too  small,  informa :ion  for  the  estimation  of  difficulties  for  middle 
level  items  may  be  provided,  but  information  for  estimating  their 
discrimination  may  not  be.  Information  for  the  estimation  of 
discriminations  for  easy  and  hard  items  may  be  provided,  but 
information  for  estimating  their  difficulties  may  not  be. 

If  the  3PL  is  the  most  appropriate  model,  only  abilities  well 
below  the  item  difficulty  are  informative  about  c.  Abilities  below 
and  above  the  difficulty  are  most  informative  about  a  and  abilities 
slightly  above  the  difficulty  are  most  informative  about  b.  Even  if 
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all  items  are  of  equal  difficulty,  a  normal  ditribution  of  abilities 
centered  slightly  above  the  item  difficulty  may  not  provide  much 
information  for  the  estimation  of  all  parameters  simultaneously.  If 
the  items  have  a  spread  of  difficulties,  better  results  may  be 
obtained  by  sampling  all  ability  levels  equally.  Wingersky  and  Lord 
(1984)  show  that  when  item  and  ability  parameters  are  estimated 
simultaneously,  a  sample  of  abilities  drawn  from  a  uniform 
distribution  produces  standard  errors  nearly  as  small  as  a  sample  of 
abilities  four  times  as  large  drawn  from  a  bell-shaped  distribution. 

Calibration  samples,  particularly  for  the  more  complex  models, 
typically  consist  of  several  thousand  examinees.  Depending  upon  the 
nature  of  the  collection  of  items  to  be  calibrated,  which  can  be 
roughly  assessed  through  the  use  of  conventional  item  statistics, 
such  samples,  although  large,  may  not  prove  useful  for  estimating 
the  paramters  of  all  items.  If  the  success  of  a  particular 
application  of  IRT  depends  heavily  on  the  veracity  of  item  level 
data,  it  seems  worthwhile  to  consider  selecting  more  informative 
samples . 
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Table  1 

Optimum  Abilities  and  Maximum  Contributions  (in  Parenthesses) 
to  Information  for  Estimating  a  or  b  Assuming  Fixed  Values 
for  the  Other  Parameters  for  the  3PL  Model  with  c  =  .2 


a 

b 

lower 

optimal 

9  for  a 

upper 

optimal 

9  for  a 

optimal 
9  for  b 

.5 

-2 

-3.96 

.86 

-1.69 

(i  =  .58,  1.38) 

-1 

-2.96 

1.86 

-  .69 

aa 

0 

-1.96 

2.86 

.31 

(Sb  ”  -^2) 

1 

-  .96 

3.86 

1.31 

2 

.04 

4.86 

2.31 

1.0 

-2 

-2.98 

-  .57 

-1.84 

(i  =  .14,  .35) 

-1 

-1.98 

.43 

-  .84 

aa 

0 

-.98 

1.43 

.16 

('bb  = 

1 

.02 

2.43 

1.16 

2 

1.02 

3.43 

2.16 

1.5 

-2 

-2.65 

-1.05 

-1.90 

(i  =  .06,  .15) 

-1 

-1.65 

-.05 

-.90 

aa 

0 

-  .65 

.95 

.10 

(ibb  =  1.11) 

1 

.35 

1.95 

1.10 

2 

1.35 

2.95 

2.10 

2.0 

-2 

-2.49 

00 

CM 

-1.92 

(i  =  .04,  .09) 

-1 

-1.49 

-  .28 

-  .92 

aa 

0 

-  .49 

.72 

.08 

(ibb  = 

1 

.51 

1.72 

1.08 

2 

1.51 

2.72 

2.08 

2.5 

(i_  =  .02,  .06) 
aa 


(i 


bb 


2 

-2.39 

-1.43 

-1.94 

1 

-1.39 

-  .43 

-  .94 

0 

-.39 

.57 

06 

1 

.61 

1.57 

1.06 

2 

1.61 

2.57 

2.06 

3.08) 


; 
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Table  2 

Maximum  Contributions  (in  Parentbesses)  to 

Information  for  Estimating  a  or  b  Assuming  Fixed  Values 

for  the 

Other  Parameters 

for  the 

2PL  and  IPL  Models  with  c 

-  0 

a 

b 

lower 

optimal 

6  for  a 

upper 

optimal 

6  for  a 

optimal 

6  for  b 

.5 

-2 

*4.82 

.82 

-2.00 

(i  =  1.76,  1.76) 

-1 

-3.82 

1.82 

-1.00 

aa 

0 

-2.82 

2.82 

.00 

(Sb  = 

1 

-1.82 

3.82 

1.00 

2 

-  .82 

4.82 

2.00 

1.0 

-2 

-3.41 

-  .59 

-2.00 

(i  =  .44,  .44) 

-1 

-2.41 

.41 

-1.00 

aa 

0 

-1.41 

1.41 

.00 

('bb  =  -^2) 

1 

-  .41 

2.41 

1.00 

2 

.59 

3.41 

2.00 

1.5 

-2 

-2.94 

-1.06 

-2.00 

(i  =  .20,  .20) 

-1 

-1.94 

-  .06 

-1.00 

aa 

0 

-  .94 

.94 

.00 

(ibb  =  1.63) 

1 

.06 

1.94 

1.00 

2 

1.06 

2.94 

2.00 

2.0 

-2 

-2.70 

-1.30 

-2.00 

(i  =*  .11,  .11) 

-1 

-1.70 

-  .30 

-1.00 

aa 

0 

-  .70 

.70 

.00 

(ibb  =  2.90) 

1 

.30 

1.70 

1.00 

2 

1.30 

2.70 

2.00 

2.5 

-2 

-2.56 

-1.44 

-2.00 

(i  =  .07,  .07) 

-1 

-1.56 

-  .44 

-1.00 

aa 

0 

-  .56 

.56 

.00 

(i^b  “  ^-53) 

1 

.44 

1.56 

1.00 

2 

1.44 

2.56 

2.00 

Probability  Probability 


Figure  1.  The  top  panel  shows  P  as  a  function  of  0  ~  b  from  Equation 
9  for  the  3PL  (solid  curves).  The  bottom  panel  shows  P  as 
a  function  of  0  —  b  from  Equation  12  for  the  2PL  (solid 
curves). 


Figure  4.  Standard  errors  of  estimated  item  difficulties  for  the  3PL 
model,  a=t5,  c=:0.  A  calibration  sample  of  N=2,500  is 
drawn  from  an  N(0,1)  distribution  (solid)  and  from  an 
N(0,4)  distribution  (dotted). 
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University  of  Iowa 
I  owa  >-  i  tv ,  1 A  b?.242 

Dr.  Ricfiard  L,  Ferguson 
American  College  Testing 
P.O.  Box  IBB 
I owa  City,  1 A  52243 

Dr.  Gerhard  Fischer 
L  I  eh  I  gqasse  5/3 
A  1010  Vienna 
AUSTRIA 

Dr .  Myron  F i sch I 
iJ.S.  Army  Headquarters 
DAHE-MPP 
The  Pentagon 

Wasninglon,  DC  20310-0300 

Prof.  Donald  Fit/geraid 
University  of  New  Enoland 
Department  of  Psychology 
Armidale,  New  South  Wales  2351 
AUSTRALIA 

Mr.  f^aul  Foley 

Navy  Personnel  RXD  Center 

San  Diego.  CA  92152-6300 

Dr  .  A  ■'  f  red  R.  F  reo  I  y 
AFObR/NL  ,  Bldg.  410 
Bolling  APB,  DC  20332-6448 

Dr.  Robert  D.  Gibbons 
Illinois  State  Psychiatric  Inst. 
Rm  529W 

1601  W.  Taylor  Street 
Chicago.  IL  60612 

Dr .  Janice  Gifford 
University  of  Massachusetts 
School  of  Education 
Amherst,  MA  01003 


Dr.  Robert  Glaser 
Learning  Research 

&  Development  Cer'ter 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15260 

Dr.  Bert  Green 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  &  34th  Street 
Baltimore,  MD  21218 

DORNIER  GMBH 

P.O.  box  1420 

D-7990  Fr i edr I chshaf en  1 

WEST  GERMANY 

Dr.  Ronald  K.  Hambleton 
University  of  Massachusetts 
Laboratory  of  Psychometric 
and  Evaluative  Researen 
Hills  South .  Room  152 
Amherst.  MA  01003 

Dr .  De I wyn  Harn i sch 
University  of  Illinois 
51  Gertv  Drive 
Champaign.  IL  LTIB2O 

Dr .  Gran t  Henn 1 ng 
Senior  Research  Scientist 
Division  of  Measurement 
Rcsearcii  and  Services 
Educational  Testing  Service 
Princeton.  NJ  03541 

Ms.  Rebecca  Hetter 

Navy  Fersonnel  R&D  Center 

Code  63 

San  Diego,  CA  92152-6800 

Dr  .  Pau I  W .  Holland 
Educational  Testing  Service.  21-T 
Rosedale  Road 
Princeton,  NJ  08541 

I'rof  .  Lutz  P  .  Horn!  e 
Institut  fur  Psychoiogie 
RWTH  Aachen 
Jaegerstrasse  17/19 
0-5100  Aachen 
WEST  GERMANY 
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Dr.  Paul  Horst 
677  G  Gtreet,  #184 
Chula  Vista,  CA  92010 

Mr.  Dick  Hoshaw 
OP- 136 

Aril ngton  Annex 
Room  2834 

Washington,  DC  20350 

Dr.  Llovd  Humphreys 
University  of  Illinois 
Department  of  Psychology 
603  East  Daniel  Street 
Champaign,  IL  61820 

Dr.  Steven  Hunka 
3-104  E  d  u  c .  N . 

University  of  Alberta 
Edmonton,  Alberta 
CANADA  T6G  2G5 

Dr.  Huvnh  Huvnh 
Coll  ege  of  Lduca  t i on 
Univ.  of  South  Carolina 
C 0  I umb i a ,  SC  29208 

Dr.  Robert  Jannarone 
Elec,  and  Computer  Eng.  Dept. 
University  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Douglas  H.  Jones 
Thatcher  Jones  Associates 
P.O.  Box  6640 
10  T  raf  a  I  gar  Court 
L awrence V i I  I e ,  NJ  08648 

Dr.  Mil  ton  S.  Katz 
European  Science  Coordination 
Office 

U.S.  Army  Research  Institute 
Box  65 

FPO  New  York  09510-1500 

Prof.  John  A.  Keats 
Department  of  Psveho I oav 
L  -iiversitv  of  Newcastle 
N.S.W.  230b 
AUS1RALIA 


Dr.  G.  Gage  Kingsbury 

Portland  Public  Schools 

Research  and  Evaluation  Department 

501  North  Dixon  Street 

P.  0.  Box  3107 

Portland,  OR  97209-3107 

Or.  William  Koch 
Box  7246,  Meas.  and  Eval.  Ctr. 
University  of  Texas-Austin 
Austin,  TX  78703 

Dr.  James  Kraatz 
Computer-based  Education 
Research  Laboratory 
University  of  Illinois 
Urbana,  IL  61801 

Dr.  Leonard  Kroeker 
Navy  Personnel  R&D  Center 
Code  62 

San  Diego,  CA  92152-6800 

Or.  Jerry  Lehnus 

Defense  Manpower  Data  Center 

Suite  400 

1600  Wi I  son  B I vd 

Rosslyn,  VA  22209 

Dr.  Thomas  Leonard 
University  of  Wisconsin 
Department  of  Statistics 
1210  West  Dayton  Street 
Madison,  WI  53705 

Or.  Michael  Levine 
Educational  Psychology 
210  Educat i on  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  Charles  Lewis 
Educational  Testinp  Service 
Princeton,  NJ  08541-0001 

Dr .  Robert  1. .  Linn 
Campus  hox  24^ 

University  of  Colorado 
Boulder,  CO  80309-0249 
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Ur  .  K'.f.e  r  i  l  oc  k  inan 
Len-l-r  ’or  Naval  Analysis 
4401  Ford  Avenue 
P.O.  Box  16268 
Alexandria,  VA  22802-0268 

Dr.  Frederic  M.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  George  B.  Macready 
Department  of  Measurement 
Statistics  &  Evaluation 
College  of  Education 
University  of  Maryland 
Col  lege  Park,  MD  20/42 

Dr.  Gary  Marco 
Stop  81-E 

Educational  Testing  Service 
Princeton,  NJ  08451 

Dr.  James  P.  McBride 
The  Psychological  Corporation 
1 250  Sixtn  Avenue 
San  Diego,  CA  92101 

Dr.  Clarence  C,  McCormick 
HO,  USMEPCOM/MLPCT 
2500  Green  Bay  Road 
North  Chicago,  IL  b0064 

Dr.  Robert  McKinley 
Educational  Testing  Service 
16-T 

Princeton,  NJ  08541 

Dr.  JaiTies  McMichael 
Technical  D i recto r 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 

Dr.  Barbara  Means 
SRI  International 
333  Ravcnswood  Avenue 
Menlo  Park,  CA  94025 

Dr .  Robert  M i s I evy 
Educational  Testirig  Service 
Princeton,  NJ  08541 


L)r.  William  Montague 

NPPDC  Code  13 

San  Diego,  CA  92152-6800 

Ms.  Kathleen  Moreno 
Navy  Personnel  R&D  Center 
Code  62 

Sari  Diego,  CA  92152-6800 

Headquarters  Marine  Corps 
Code  MPI-20 
Washington,  DC  20360 

Ur.  W.  Alan  Nicewander 
University  of  Oklahoma 
Department  of  Psychology 
Norman,  OK  73071 

Deputy  Technical  Director 

NPRDC  Code  OlA 

San  Diego,  CA  92152-6800 

Director,  Trainirig  Laboratory, 
NPRDC  (Code  05) 

San  Diego,  CA  92152-6800 

Director,  Manpower  and  Personnel 
Laboratory , 

NPRDC  (Code  06) 

San  Diego,  CA  92152-6600 

Director,  Human  Factors 

&  Organizational  Svstcnis  Lab, 
NI’RDC  (Code  07) 

San  Diego,  CA  92152-6800 

L  I  brary ,  NPRDC 
Code  P201L 

San  Diego,  CA  92152-6800 

Commanding  Officer, 

Naval  Research  Laboratory 
Code  2627 

Washington,  DC  20390 

O'-.  Harold  F.  O'Neil,  Jr. 

School  of  Education  -  WPH  801 
Department  of  Educational 
Psychology  8.  Fecfinologv 
University  of  Southern  California 
Los  Angeles,  CA  90089-0031 
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800  N.  ljuincv  Street 
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C.:de 

6.0  N.  Ouincv  Street 
Arlington,  VA  2221/~S000 

Assist  ant  for  M*-' I  Rt  search, 

''■pvp  .apfl.pn-j  a 'id  StU'jier 

vP  01b' 

Washington,  UU  203/0 

Or.  Judith  1 .1  r  a  s  a  n  u 
Basic  K'fsearcti  Ufl  ice 
A iTi  V  P  e  i  ir  V  r  c  h  Institute 
5001  Eiceni-iower  Avpnue 
Ale>an.jrid.  VA  22333 

Dr.  Jesse  Orlansky 

Institute  for  Defense  Analyses 

1801  N.  Beauregard  St. 

Ale>ianoria,  VA  22311 

Dr.  Randolph  Park 
Armv  Rc'searcfi  Institute 
5001  Eisenhower  Blvd. 

Alexandria,  VA  22333 

Wayne  M.  Patience 
American  Council  on  E.iJu cation 
GED  Testing  Seryice,  Suite  20 
One  Dupont  Circle,  NW 
Washington,  DC  2003G 

Dr .  James  Pau  I  son 
Department  of  Psivchology 
Portland  State  University 
P.O.  Box  751 
Portland,  OR  9720V 

Dept,  of  Administrative  Sciences 
Code  54 

Naval  Postgraduate  School 
f^onterey,  CA  93943-5026 


Department  oi  Operatioris  Research. 

Naval  Postgraduate  Sctiool 
Monterev,  CA  93940 

Dr.  Hark  D.  Reckase 
ACT 

P.  0.  Box  168 
Iowa  City,  I A  52243 

Ur.  Mai  CO  I m  Ree 
APHRC/MOA 

Brooks  ARB.  TX  /B235 

Dr.  Barry  Riegelhaupt 
HumRRO 

1100  South  Washington 
Alexandria,  V  A  2  2  c 1 4 

Dr.  Carl  Ross 
CNLT -PDCD 
Bui  I  d  i  ng  90 

Great  Lakes  NTC,  it  C'.^OoS 
Dr.  J .  Ryan 

Department  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Pumiko  Sameiima 
Department  of  Psychology 
University  of  lennessee 
310B  Austin  Peay  Bldg. 

Rrioxvi  I  le.  TN  3/918-0900 

Mr.  Drew  Sands 

Nf-’RDC  Code  62 

San  Diego,  LA  92152-8)000 

Lowe  I  1  Schoer 

Psychological  &  Duant i tat  1 ve 
r  ounda t i ons 
College  of  Education 
University  of  1 owa 
I owa  City,  I A  52242 

Dr.  Mary  Schratz 

Navv  Personnel  Ri^D  Center 

San  Diego,  CA  92152  EciOO 

Dr .  Dan  Segal  1 

Navy  Personnel  Rt»D  Center 

San  Diego,  CA  92152 
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[t  .  W.  Steve  Se I  I  man 
IjASSiMRAk.L) 

2621.59  The  l-’entagon 
Washington,  DC  20301 

Dr.  Ka7uo  Shigemasu 
7-9-24  Kugenuma-Ka i gan 
Fujisawa  251 
JAPAN 

Dr .  W  M I  i am  S i ms 
Center  tor  Naval  Analysis 
4401  Ford  Avenue 
P.O.  Box  1B2B8 
Alexandria,  VA  22302-02B8 

Dr.  H.  Wallace  Sinaiko 
Manpower  Research 

and  Advisory  Services 
Smithsonian  Institution 
801  North  Pitt  Street,  Suite  120 
Alexandria,  VA  22314-1713 

Dr.  Richard  E.  Snow 
Schoo  I  Of  Educa  1 1 on 
Stanford  University 
Stanford.  LA  94305 

Dr.  Richard  C.  Sorensen 
Navy  Personnel  RSD  Center 
San  Diego,  CA  92152-6600 

Dr .  Paul  Speckman 
University  of  Missouri 
Department  of  Statistics 
Co  I umb I  a ,  MU  65201 

Dr.  Judy  Spray 
ACT 

P.O.  Box  1 bb 
I owa  C  I  tv ,  I A  52243 

Dr.  Martfia  Stocking 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr .  William  Stout 
University  of  Illinois 
Department  of  Statistics 
101  I  I  I  ini  Hall 
725  South  Wright  St. 

Champaign,  IL  61820 


Dr.  Hariharan  Swani  i  nat  fian 
Laboratory  of  Psychometric  and 
Evaluation  Research 
Schoo  I  of  Educat  i  on 
University  of  Massachusetts 
Amherst,  MA  01003 

Mr.  Brad  Sympson 

Navy  Personnel  R&D  Center 

Code-62 

San  Diego,  CA  92152-6800 

Dr.  John  I  a  n  q  n  e  y 

AFOSR/NL,  Bldg.  410 

Eio  I  ling  AF  B .  DC  20332-6448 

Dr .  K i kum i  T a t s uoka 
CERE 

252  Lnqineoring  Research 
Laboratory 

103  S.  Mathews  Avenue 
Urbana,  IL  61801 

Dr.  Maurice  Tatsuoka 
220  Educa  I,  ion  Bldg 
1310  S .  Sixth  St. 

Champaign,  IL  61820 

Dr ,  David  T  h  i  ssen 
Department  of  Psychology 
University  of  Kansas 
Lawrence,  KS  66044 

Mr.  Gary  Thomasson 
University  of  Illinois 
Educational  F^sychology 
Champaign,  IL  61C20 

Dr.  Robert  Tsutakawa 
University  of  Missouri 
Department  of  Statistics 
c;22  Math.  Sciences  Bldg. 

Co  I  umb  1  a ,  MO  LT521 1 

Dr.  Ledyard  Tucker 
University  of  Illinois 
Llepartment  of  h'sychology 
603  L.  Daniel  Street 
Champa  I  on,  IL  61320 
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Dr.  Venn  M.  Lirry 
Perbonnel  PiD  Center 
Office  ot  Ptrsorifiel  Management 
19t'0  E.  Street,  NW 
Washington,  OC  20415 

Dr .  Dav id  Vale 
Assessment  Systems  Corp. 

2233  University  Avenue 
bu  .  t-  44() 

St.  Paul,  Mtvi  55114 

Dr.  frank  L.  Vicino 
Maw  Personnel  PiyO  Center 
San  Diego,  CA  32152-6300 

Dr .  Howard  Wa i ne  r 
Education;-.  I  Testing  Service 
Princeton,  Nl  08541 

Dr.  Ming-Mei  Wang 
Cindouist  Center 
t  .•  r  M  f  a  s  u  r  f  ■  rti  e  n  t 
U  n  I  e  r  I  1  y  c- 1  i  0  w  a 
1 owa  City,  1 A  52242 

Dr.  1 homas  A.  Warm 
Coast  Guard  Institute 
P.  J.  SuCstatiori  18 
Oklahoma  City,  OK  73169 

Dr.  Brian  Waters 
HumRPO 

12908  Argvle  Circle 
A  1  exaridr  i  a  ,  VA  22314 

Iir .  David  J .  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E .  River  Road 
Minneapolis,  MN  55455-0344 

Dr.  Ronald  A.  Weitznian 
Box  146 

Carme I  ,  CA  93921 


Dr.  Douglas  Wetzel 
Code  5l 

Navy  Personnel  RliiJ  tenter 
ban  Diego,  LA  92152-6800 

Dr.  Rand  R.  Wilcox 
University  of  Southern 
Cal  1 f  0  r  n i a 

Department  of  Psychology 
Los  Angeles,  CA  90059-1061 

German  Military  Representative 
A1IN:  Wolfgang  Wildorube 
Stre I tkraef  teamt 
D-CbOU  Bonn  2 

4000  Brandywine  btreet,  NW 
Wasti  I  ngton  ,  DC  20016 

Or .  Bruce  Willi ams 
Department  of  Lducat  onal 
Psvcho  I  ogy 

University  of  Illinois 
Ur ban a,  IL  61801 

Dr .  Hilda  Wing 
NRC  MH-176 

2101  Constitution  Ave. 
Washington,  DC  20418 

Ur .  Martin  F .  W i skof f 
Defense  Manpower  Data  Center 
550  C amino  El  Estero 
bu I tc  200 

Monterey,  CA  93943-3231 

Mr.  John  H.  Wolfe 

Navy  Personnel  RiD  Center 

San  Diego,  CA  92152-6800 

Dr.  George  Wong 
biostatistics  Laboratory 
Memorial  SI oan-Ketter i ng 
Cancer  Center 
12/5  York  Avenue 
New  York,  NY  10021 


Major  John  Welsh 
4FHRL /MOAN 

Brooks  AFB,  TX  70223 


Dr.  Wallace  Wulfeck,  III 
Navv  Personnel  Rfl.L')  tenter 
Code  51 

San  Diego,  CA  92152-6800 
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Dr.  Kentaro  Yamamoto 
03-T 

Educational  Testing  Service 
Rosedale  Road 
Princeton,  NJ  08541 

Dr-.  Wendy  Yen 
CTB/McGraw  Hill 
Del  Monte  Research  Park 
Monterey,  CA  93940 

Dr.  Joseph  L.  Young 
National  Science  Foundation 
Room  320 

1800  G  Street,  N.W. 
Washington,  DC  20550 

Mr.  Anthony  R.  Zara 
National  Council  of  fjtate 
Boards  of  Nursing,  Inc. 
625  North  Michigan  Avenue 
Su i te  1544 
Chicago.  !L  60611 

Dr.  f-'  e  t  e  r  S  t  o  1  o  f  f 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.O.  Box  16268 
Alexandria,  VA  22302-0263 


